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2.  Objectives: 


The  overall  goal  of  the  research  proposed  here  is  to  develop  a  framework  for  controlled  mobility 
in  ad-hoc  networks  that  integrates  data  ferrying  in  sparse  or  delay  tolerant  networks  with  local 
decentralized  control  for  improved  real-time  network  flow.  Specific  objectives  to  be  completed 
as  part  of  this  work  include: 

01.  Define  Performance  Metrics:  Investigate  metrics  for  understanding  the  effects  of  controlled 
mobility  on  ad-hoc  networks.  These  metrics  will  be  functions  of  network  objectives  such  as 
packet  delay  and  throughput;  mobile  node  capabilities  such  as  speed  and  buffer  size;  and  client 
node  behavior  such  as  speed  and  spatial  density. 

02.  Design  Ferry-Based  Communication  Algorithms:  Develop  and  evaluate  controlled  mobility 
algorithms  for  data  ferries  in  sparse  or  delay  tolerant  networks.  Multiple  mobility  control 
algorithms  will  be  designed  and  evaluated  against  the  metrics  defined  in  01.  Algorithms  will  be 
identified  that  have  superior  performance  in  different  scenarios. 

03.  Integrate  Realistic  Communication  Models:  Design  feedback  controllers  and  transmission 
protocols  that  optimize  data  ferrying  given  a  continuous  model  of  communication  rate  versus 
separation  distance.  Standard  wireless  communication  models  use  the  simple  assumption  that 
communication  is  constant  up  until  some  specified  separation  distance.  In  reality  wireless 
communication  channels  are  richer  than  this,  with  the  channel  capacity  set  based  on  the  ratio  of 
measured  signal  to  noise  power. 

04.  Develop  Ferry  Allocation  Algorithms:  Develop  linear  programming  and  reinforcement  learning 
techniques  for  optimal  allocation  of  data  ferries  in  multicast  networks.  Unknown  motion  of  task 
nodes  in  an  ad-hoc  network  will  be  addressed  by  the  development  of  reinforcement  learning 
algorithms  that  adapt  the  data  ferrying  trajectories  as  the  network  evolves. 

05.  Decentralized  Cooperative  Control  for  Communication:  Analyze  decentralized  mobility 
control  algorithms  to  improve  the  performance  of  real-time  traffic  flow  through  multi-hop  links  in 
the  presence  of  unmodcled  interference.  Real-time  data  relaying  will  be  recast  as  a  decentralized 
control  problem  in  which  node  mobility  is  directly  tied  to  network  performance  metrics  such  as 
capacity. 

06.  Decentralized  Cooperative  Control  for  Other  Tasks:  Evaluate  distributed  multi-objective 
control  algorithms  for  networks  of  mobile  nodes  capable  of  carrying  out  primary  tasks  such  as 
surveillance  while  also  acting  as  data  relays  in  an  ad-hoc  network.  The  decentralized  control 
algorithms  developed  in  05  will  be  augmented  to  include  additional  objectives. 

07.  Demonstrate  Integrated  Concept:  Demonstrate  the  integration  of  data  ferrying  (developed  in 
02-04)  with  decentralized  mobility  control  for  real-time  flow  (05-06)  in  a  single  controlled 
mobility  framework  based  on  reinforcement  learning  and  local,  decentralized  control. 
Simulations,  laboratory  experiments,  and  field  deployments  on  the  Ad-hoc  UAV  Ground 
Network  (AUGNct)  at  CU  Boulder  will  verify  the  concepts  proposed  here. 


3.  Status  of  Effort: 

We  completed  objectives  01  -  05.  Objective  06  was  not  achieved.  Flight  testing  for  objective 
07  partially  demonstrated  01-05. 


4.  Major  accomplishments: 


1 .  The  notion  of  a  quality  of  service  phase  space  (Fig.  1)  was  developed  to  show  what 
communication  modes  (Direct,  Relay,  or  Ferry)  can  provide  a  given  QoS  demand  in 
terms  of  insertion  delay  and  average  throughput  [C.18,  J.3]. 
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Figure  1.  Phase  space  showing  best  communication  modes  for  three-node  network  with  source  and 
destination  nodes  separated  by  a.)  50  m;  b.)  5000  m;  c.)  50  km;  and  d.)  500  km. 


2.  It  was  shown  that  a  mobile  node  can  be  controlled  to  find  the  optimal  relay  point  in  a 
chain  of  relays  using  only  local  information  (Fig.  2).  The  decentralized  controller  allows 
the  network  to  self  organize,  self  adapt  when  relays  fail  or  noise  sources  are  introduced, 
and  tether  to  moving  source/dcstination  nodes.  The  underlying  control  theory  has  been 
developed  and  we  are  able  to  show  stability  of  the  decentralized  system  given  several 
different  link  and  network  models  [C.2,  J.5]. 
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Figure  2.  Simulation  of  a  pianar  cascaded  relay  network  using  two  point-mass  robotic  relay  nodes  in  a 
flow-pipe  network  using  the  Shannon  channel  capacity  model.  At  time  t  =  250  s  a  jamming  noise  source 
is  introduced  showing  the  ability  of  the  gradient  controller  to  react  to  changes  in  the  RF  environment 
and  improve  the  capacity  above  that  of  a  range-based  solution,  (a)  Node  positions  in  the  environment 
and  the  path  of  the  relays.  The  diamond  markers  represent  the  location  of  the  range-based  solution,  (b) 
Capacity  of  the  chain  using  the  gradient  controller  compared  to  that  obtained  from  a  range-based 
solution,  along  with  the  direct  link  capacity.  Note  that  the  range-based  solution  has  a  lower  capacity  than 
using  direct  communication  from  the  source  to  the  destination. 
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3.  An  extremum  seeking  control  architecture  for  electronic  leashing  and  chaining 
applications  was  developed  that  combines  vector  field  guidance,  stochastic 
approximation,  and  gradient  descent  [J.l,  T.l,  C.2,  J.6]. 


Ad  Hoc  Network 


Figure  3.  An  extremum-seeking  architecture  for  a  linked  network  chain  of  mobile  relay  nodes 
in  2D.  For  a  non-holonomic  vehicle  such  as  an  unmanned  aircraft,  an  orbit-tracking 
controller  provides  the  dither  signal  needed  to  estimate  the  field  gradient. 


4.  The  extremum  seeking  control  architecture  was  applied  to  electronic  leashing  (finding  a 
single  radio  emitting  source)  and  chaining  (forming  an  optimal  cascade  relay  network)  in 
simulation  and  experiment.  Simulation  results  (Fig.  4)  verified  the  stability  of  the  control 
approach  for  electronic  chaining  while  experimental  flight  results  were  obtained  to 
demonstrate  leashing  (Fig.  5)  [J.l,  T.l,  C.2,  J.6]. 
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Figure  4.  Simulation  of  three  (3)  UA  relay  nodes  reacting  to  a  localized  noise  source,  (a)  Motion  of  UAs 
within  the  environment  also  showing  noise  source  location  and  the  SNR  contours  of  the  two  end  nodes. 

(b)  The  minimum  SNR  value  along  the  chain  during  the  simulation. 
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Figure  5.  Experimental  data  from  one  UA  leashed  to  a  stationary  ground  nodes,  (a)  Motion  of  UA  within 
the  environment  also  showing  RF  source  location,  (b)  Measured  signal  strength  over  time. 

5.  Ferrying  route  design  using  reinforcement  learning  has  been  developed  and  demonstrated 
in  simulation  (Fig.  6).  It  has  been  shown  to  exceed  the  performance  of  competing 
methods  [T.2,  C.4]. 


TSP  =  Traveling  Salesman  solution  RR  =  Round  Robin  (naive) 
RL  =  Reinforcement  Learning  STO  =  Stochastic  Modeling 


Figure  6.  Performance  of  Reinforcement  Learning  path  planning  compared  to 
competing  methods  on  four  different  scenarios. 


6.  A  DTN  protocol  has  been  developed  to  support  data  ferrying.  In  flights  with  unmanned 
aircraft  (Fig.  7)  it  was  shown  to  significantly  increase  the  fraction  of  packets  delivered 
compared  to  direct  and  relay  communication  [C.3]. 


Figure  7.  Comparison  of  different 
protocols’  performances  on  actual 
unmanned  aircraft  flights.  Ad  hoc 
relaying  using  fixed  nodes  on  the  ground 
reduces  packet  loss  from  35%  to  22%. 
Adding  a  delay  tolerant  network  protocol 
on  top  of  the  ad  hoc  network  further 
reduces  packet  loss  to  10%.  The  figure  at 
right  shows  the  experimental  set-up  on 
our  test  range. 
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The  overall  goal  of  this  research  project  is  to  develop  a  framework  for 
controlled  mobility  in  ad-hoc  networks  that  integrates  data  ferrying  in 
sparse  or  delay  tolerant  networks  with  local  decentralized  control  for 
improved  real-time  network  flow.  The  original  Statement  of  Work  still 
holds . 

We  have  completed  objectives  01-05.  Flight  testing  of  the  algorithms 
developed  to  meet  these  will  be  conducted  in  Summer  2008.  Major 
accomplishments  this  past  year  include  analysis  of  decentralized 
control  for  optimizing  capacity  of  linear  cascade  network  chains  and 
designing  data  ferry  routes  using  reinforcement  learning  algorithms 
that  outperform  standard  Traveling  Salesperson  and  stochastic 
programming  approaches.  Five  archival  publications  have  been  generated 
(1  published,  2  accepted  and  3  under  review)  through  this  effort. 
Aspects  of  this  project  continue  to  transition  into  work  funded  by  L3 
Communication  and  USAF  Big  Safari  Special  Group. 
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